Test-time training

A technique of doing some model training (e.g., Gradient descent) during test-time. See also Test time scaling

A nice way to think about this is considering the three steps: Pre training to obtain a general understanding of language/visual/… system. Finetuning and prompt tuning to specialize, and then test-time training to a specific problem.

Sun2019testtime proposed the test-time training.

Akyürek2024surprising shows effectiveness in abstract Reasoning.